Unsupervised clinical relevancy ranking of structured medical records to retrieve condition-specific information in the emergency department.

BACKGROUND: Decision making in the Emergency Department (ED) requires timely identification of clinical information relevant to the complaints. Existing information retrieval solutions for the electronic health record (EHR) focus on patient cohort identification and lack clinical relevancy ranking. We aimed to compare knowledge-based (KB) and unsupervised statistical methods for ranking EHR information by relevancy to a chief complaint of chest or back pain among ED patients. METHODS: We used Pointwise-mutual information (PMI) with corpus level significance adjustment (cPMId), which modifies PMI to reward co-occurrence patterns with a higher absolute count. cPMId for each pair of medication/problem and chief complaint was estimated from a corpus of 100,000 un-annotated ED encounters. Five specialist physicians ranked the relevancy of medications and problems to each chief complaint on a 0-4 Likert scale to form the KB ranking. Reverse chronological order was used as a baseline. We directly compared the three methods on 1010 medications and 2913 problems from 99 patients with chest or back pain, where each item was manually labeled as relevant or not to the chief complaint, using mean average-precision. RESULTS: cPMId out-performed KB ranking on problems (86.8% vs. 81.3%, p < 0.01) but under-performed it on medications (93.1% vs. 96.8%, p < 0.01). Both methods significantly outperformed the baseline for both medications and problems (71.8% and 72.1%, respectively, p < 0.01 for both comparisons). The two complaints represented virtually completely different information needs (average Jaccard index of 0.008). CONCLUSION: A fully unsupervised statistical method can provide a reasonably accurate, low-effort and scalable means for situation-specific ranking of clinical information within the EHR.

A dynamic reaction picklist for improving allergy reaction documentation in the electronic health record.

OBJECTIVE: Incomplete and static reaction picklists in the allergy module led to free-text and missing entries that inhibit the clinical decision support intended to prevent adverse drug reactions. We developed a novel, data-driven, "dynamic" reaction picklist to improve allergy documentation in the electronic health record (EHR). MATERIALS AND METHODS: We split 3 decades of allergy entries in the EHR of a large Massachusetts healthcare system into development and validation datasets. We consolidated duplicate allergens and those with the same ingredients or allergen groups. We created a reaction value set via expert review of a previously developed value set and then applied natural language processing to reconcile reactions from structured and free-text entries. Three association rule-mining measures were used to develop a comprehensive reaction picklist dynamically ranked by allergen. The dynamic picklist was assessed using recall at top k suggested reactions, comparing performance to the static picklist. RESULTS: The modified reaction value set contained 490 reaction concepts. Among 4 234 327 allergy entries collected, 7463 unique consolidated allergens and 469 unique reactions were identified. Of the 3 dynamic reaction picklists developed, the 1 with the optimal ranking achieved recalls of 0.632, 0.763, and 0.822 at the top 5, 10, and 15, respectively, significantly outperforming the static reaction picklist ranked by reaction frequency. CONCLUSION: The dynamic reaction picklist developed using EHR data and a statistical measure was superior to the static picklist and suggested proper reactions for allergy documentation. Further studies might evaluate the usability and impact on allergy documentation in the EHR.

Facilitating information extraction without annotated data using unsupervised and positive-unlabeled learning.

Information extraction (IE), the distillation of specific information from unstructured data, is a core task in natural language processing. For rare entities (<1% prevalence), collection of positive examples required to train a model may require an infeasibly large sample of mostly negative ones. We combined unsupervised- with biased positive-unlabeled (PU) learning methods to: 1) facilitate positive example collection while maintaining the assumptions needed to 2) learn a binary classifier from the biased positive-unlabeled data alone. We tested the methods on a real-life use case of rare (<0.42%) entity extraction from medical malpractice documents. When tested on a manually reviewed random sample of documents, the PU model achieved an area under the precision-recall curve of0.283 and Fj of 0.410, outperforming fully supervised learning (0.022 and 0.096, respectively). The results demonstrate our method's potential to reduce the manual effort required for extracting rare entities from narrative texts.

Mining clinical phrases from nursing notes to discover risk factors of patient deterioration.

OBJECTIVE: Early identification and treatment of patient deterioration is crucial to improving clinical outcomes. To act, hospital rapid response (RR) teams often rely on nurses' clinical judgement typically documented narratively in the electronic health record (EHR). We developed a data-driven, unsupervised method to discover potential risk factors of RR events from nursing notes. METHODS: We applied multiple natural language processing methods, including language modelling, word embeddings, and two phrase mining methods (TextRank and NC-Value), to identify quality phrases that represent clinical entities from unannotated nursing notes. TextRank was used to determine the important word-sequences in each note. NC-Value was then used to globally rank the locally-important sequences across the whole corpus. We evaluated our method both on its accuracy compared to human judgement and on the ability of the mined phrases to predict a clinical outcome, RR event hazard. RESULTS: When applied to 61,740 hospital encounters with 1,067 RR events and 778,955 notes, our method achieved an average precision of 0.590 to 0.764 (when excluding numeric tokens). Time-dependent covariates Cox model using the phrases achieved a concordance index of 0.739. Clustering the phrases revealed clinical concepts significantly associated with RR event hazard. DISCUSSION: Our findings demonstrate that our minimal-annotation, unsurprised method can rapidly mine quality phrases from a large amount of nursing notes, and these identified phrases are useful for downstream tasks, such as clinical outcome predication and risk factor identification.

Unsupervised Machine Learning of Topics Documented by Nurses about Hospitalized Patients Prior to a Rapid-Response Event.

BACKGROUND: In the hospital setting, it is crucial to identify patients at risk for deterioration before it fully develops, so providers can respond rapidly to reverse the deterioration. Rapid response (RR) activation criteria include a subjective component ("worried about the patient") that is often documented in nurses' notes and is hard to capture and quantify, hindering active screening for deteriorating patients. OBJECTIVES: We used unsupervised machine learning to automatically discover RR event risk/protective factors from unstructured nursing notes. METHODS: In this retrospective cohort study, we obtained nursing notes of hospitalized, nonintensive care unit patients, documented from 2015 through 2018 from Partners HealthCare databases. We applied topic modeling to those notes to reveal topics (clusters of associated words) documented by nurses. Two nursing experts named each topic with a representative Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT) concept. We used the concepts along with vital signs and demographics in a time-dependent covariates extended Cox model to identify risk/protective factors for RR event risk. RESULTS: From a total of 776,849 notes of 45,299 patients, we generated 95 stable topics, of which 80 were mapped to 72 distinct SNOMED CT concepts. Compared with a model containing only demographics and vital signs, the latent topics improved the model's predictive ability from a concordance index of 0.657 to 0.720. Thirty topics were found significantly associated with RR event risk at a 0.05 level, and 11 remained significant after Bonferroni correction of the significance level to 6.94E-04, including physical examination (hazard ratio [HR] = 1.07, 95% confidence interval [CI], 1.03-1.12), informing doctor (HR = 1.05, 95% CI, 1.03-1.08), and seizure precautions (HR = 1.08, 95% CI, 1.04-1.12). CONCLUSION: Unsupervised machine learning methods can automatically reveal interpretable and informative signals from free-text and may support early identification of patients at risk for RR events.